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METHOD FOR GENERATING AN ANIMATED 
THREE-DIMENSIONALVIDEO HEAD 

BACKGROUND OF THE INVENTION 

[0001] The present invention relates to head animation, and more particularly, to 
generating an animated three-dimensional video head based on two-dimensional 
video images. 

[0002] Virtual spaces filled with avatars are an attractive way to allow for the 
experience of a shared environment. However, animation of a photo-realistic avatar 
generally requires intensive graphic processes, particularly for rendering facial 
features. 

[0003] Accordingly, there exists a significant need for improved rendering of facial 
features. The present invention satisfies this need. 

SUMMARY OF THE INVENTION 

[0004] The present invention is embodied in a method for generating an animated 
three-dimensional video head. In the method, a sequence of two-dimensional video 
image frames of the face of an actor is captured. For each video image frame, the 
locations of the actor's facial features are sensed at predetermined node locations. 
Each node location is associated with a particular facial feature. Control points on a 
three-dimensional head mesh are driven based on the sensed node locations to 
generate a shaped three-dimensional head mesh. The video image frame used to 
generate the sensed node location is warped for projection onto the shaped head 
mesh. The warped video image frame is texture mapped onto the shaped head 
mesh to generate a three-dimensional frame head associated with the respective 
video image frame. The three-dimensional video head is animated by displaying a 
sequence of the three-dimensional frame heads associated with the sequence of 
video image frames. 

[0005] In more detailed features of the invention, the step of sensing the locations 
of the facial features in the sequence of video image frames is performed using 
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transformed facial image frames generated based on wavelet transformations, such 
as Gabor wavelet transformations. 

[0006] Other features and advantages of the present invention should be 
apparent from the following description of the preferred embodiments taken in 
conjunction with the accompanying drawings, which illustrate, by way of example, 
the principles of the invention. 

BRIEF DESCRIPTION OF THE DRAWINGS 

[0007] FIG. 1 is a flow diagram showing a technique for generating an animated 
three-dimensional video head, according with the invention. 

[0008] FIG. 2 is a schematic flow diagram showing the technique for generating 
the animated three-dimensional video head of FIG. 1, according with the invention. 

[0009] FIG. 3 is a series of image frames illustrating an animation of a three- 
dimensional video head, according to the invention. 

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS 
[0010] The present invention provides a technique for generating an animated 
three-dimensional video head based on sensed locations of facial features and 
texture mapping of corresponding two-dimensional video image frames onto a 
shaped head mesh generated using the sensed locations. The use of facial feature 
location sensing in combination with texture mapping of the same video image frame 
used in the facial feature location sensing, provides a animated three-dimensional 
video head having desirable photo-realistic properties. 

[0011] With reference to FIGS. 1 and 2, in the method for generating an animated 
video head, a sequence of video image frames 24 of the face of an actor is captured 
(step 10). For each video image frame (step 12), the locations 26 of the actor's facial 
features are sensed at predetermined node locations (step 14). Each node location 
is associated with a particular facial feature. Control points 28 on a three- 
dimensional head mesh are driven based on the sensed node locations to generate 
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a shaped three-dimensional head mesh 30 (step 16). The video image frame used 
to generate the sensed node location is warped for projection onto the shaped head 
mesh (step 18). The warped video image frame 27 is texture mapped onto the 
shaped head mesh to generate the three-dimensional frame head 32 associated with 
the respective video image frame (step 20). The three-dimensional video head is 
animated by displaying a sequence of the three-dimensional frame heads associated 
with the sequence of video image frames (step 22). 

[0012J An example of an animated three-dimensional video head is shown in FIG. 
3. A sequence of image frames 34 shows a rotating three-dimensional video head. 
The first image frame of the sequence is in the upper left-hand corner of FIG. 3 and 
the last image frame of the sequence is in the lower right-hand corner. Each image 
frame shows a two-dimensional video image frame 24 overlaid on a two-dimensional 
projection of the resulting rotating three-dimensional frame head 32. 

[0013] The step of sensing the locations of the facial features in the sequence of 
video image frames may be performed using transformed facial image frames 
generated based on wavelet transformations, such as Gabor wavelet 
transformations. 

[0014] Facial feature finding and tracking using Gabor wavelet tracking and jet 
graph matching is described in U.S. patent number 6,272,231, titled WAVELET- 
BASED FACIAL MOTION CAPTURE FOR AVATAR ANIMATION. Imaging systems 
for acquiring images and image mapping are described in U.S. patent application 
serial number 09/724,320, titled METHOD AND APPARATUS FOR RELIEF 
TEXTURE MAP FLIPPING. The entire disclosures of U.S. patent number 6,272,231 
and U.S. patent application serial number 09/724,320 are incorporated herein by 
reference. 

[0015] Although the foregoing discloses the preferred embodiments of the 
present invention, it is understood that those skilled in the art may make various 
changes to the preferred embodiments without departing from the scope of the 
invention. The invention is defined only the following claims. 
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